Graph neural networks (GNNs) have been demonstrated to be a powerful algorithmic model in broad application fields for their effectiveness in learning over graphs. To scale GNN training up for large-scale and ever-growing graphs, the most promising solution is distributed training which distributes the workload of training across multiple computing nodes. However, the workflows, computational patterns, communication patterns, and optimization techniques of distributed GNN training remain preliminarily understood. In this paper, we provide a comprehensive survey of distributed GNN training by investigating various optimization techniques used in distributed GNN training. First, distributed GNN training is classified into several categories according to their workflows. In addition, their computational patterns and communication patterns, as well as the optimization techniques proposed by recent work are introduced. Second, the software frameworks and hardware platforms of distributed GNN training are also introduced for a deeper understanding. Third, distributed GNN training is compared with distributed training of deep neural networks, emphasizing the uniqueness of distributed GNN training. Finally, interesting issues and opportunities in this field are discussed.

神经辐射场（NERF）已成功用于场景表示。最近的工作还使用基于NERF的环境表示形式开发了机器人导航和操纵系统。由于对象定位是许多机器人应用的基础，因此进一步释放了机器人系统中NERF的潜力，我们研究了NERF场景中的对象定位。我们提出了一个基于变压器的框架NERF-LOC，以在NERF场景中提取3D边界对象框。 Nerf-Loc将预先训练的NERF模型和相机视图作为输入，并产生标记为3D边界对象的框作为输出。具体来说，我们设计了一对平行的变压器编码器分支，即粗流和细流，以编码目标对象的上下文和详细信息。然后将编码的功能与注意层融合在一起，以减轻准确对象定位的歧义。我们已经将我们的方法与基于传统变压器的方法进行了比较，我们的方法可以实现更好的性能。此外，我们还提出了第一个基于NERF样品的对象定位基准Nerflocbench。

大规模图在现实情况下无处不在，可以通过图神经网络（GNN）训练以生成下游任务的表示形式。鉴于大规模图的丰富信息和复杂的拓扑结构，我们认为在这样的图中存在冗余，并将降低训练效率。不幸的是，模型可伸缩性严重限制了通过香草GNNS训练大规模图的效率。尽管在基于抽样的培训方法方面取得了最新进展，但基于抽样的GNN通常忽略了冗余问题。在大规模图上训练这些型号仍然需要无法容忍的时间。因此，我们建议通过重新思考图中的固有特征来降低冗余并提高使用GNN的大规模训练效率。在本文中，我们开拓者提出了一种称为dropreef的曾经使用的方法，以在大规模图中删除冗余。具体而言，我们首先进行初步实验，以探索大规模图中的潜在冗余。接下来，我们提出一个度量标准，以量化图中所有节点的异质性。基于实验和理论分析，我们揭示了大规模图中的冗余，即具有高邻居异质的节点和大量邻居。然后，我们建议Dropreef一劳永逸地检测并删除大规模图中的冗余，以帮助减少训练时间，同时确保模型准确性没有牺牲。为了证明DropReef的有效性，我们将其应用于最新的基于最新的采样GNN，用于训练大规模图，这是由于此类模型的高精度。使用Dropreef杠杆，可以大力提高模型的训练效率。 Dropreef高度兼容，并且在离线上执行，从而在很大程度上使目前和未来的最新采样GNN受益。

异质图神经网络（HGNN）提供了强大的能力，可以将异质图的丰富结构和语义信息嵌入到低维节点表示中。现有的HGNN通常会学习使用层次结构注意机制和重复的邻居聚集来嵌入信息，并遭受不必要的复杂性和冗余计算。本文提出了简单有效的异质图神经网络（SEHGNN），该图通过避免在相同关系中避免过度使用的节点级别的注意来降低这种过度的复杂性，并在预处理阶段预先计算邻居聚集。与以前的工作不同，Sehgnn利用轻重量参数的邻居聚合器来学习每个Metapath的结构信息，以及一个基于变压器的语义聚合器将跨Metapaths的语义信息组合为每个节点的最终嵌入。结果，SEHGNN提供了简单的网络结构，高预测准确性和快速训练速度。在五个现实世界的异质图上进行了广泛的实验，证明了Sehgnn在准确性和训练速度上的优越性。代码可在https://github.com/ict-gimlab/sehgnn上找到。

深度神经网络（DNN）已被证明是针对对抗性示例（AE）的脆弱性，这些例子是恶意设计用于欺骗目标模型的。添加了不可察觉的对抗扰动的正常示例（NES）可能是对DNN的安全威胁。尽管现有的AES检测方法已经达到了很高的精度，但他们未能利用检测到的AE的信息。因此，基于高维扰动提取，我们提出了一种无模型的AES检测方法，其整个过程没有查询受害者模型。研究表明，DNN对高维度敏感。对抗示例中隐藏的对抗性扰动属于高维特征，高维特征是高度预测性和非持胸膜的。 DNN比其他人从高维数据中学习更多细节。在我们的方法中，扰动提取器可以从AES作为高维特征提取对抗扰动，然后训练有素的AES鉴别器确定输入是否为AE。实验结果表明，所提出的方法不仅可以以高精度检测对抗示例，还可以检测AE的特定类别。同时，提取的扰动可用于将AE恢复到NES。

语音很容易泄漏，例如在不同情况下手机记录的语音。语音中的私人内容可能通过语音增强技术恶意提取。语音增强技术与深度神经网络（DNN）一起迅速发展，但是对抗性示例可能会导致DNN失败。在这项工作中，我们提出了一种对抗性方法来降低语音增强系统。实验结果表明，生成的对抗性示例可以在原始示例中删除大多数内容信息，或者通过语音增强来替换目标语音内容。在增强的原始示例和增强的对抗示例识别结果之间的单词错误率（WER）可以达到89.0％。在增强的对抗性示例和目标示例之间的目标攻击的情况低至33.75％。对抗扰动可以将变更速率带到原始示例超过1.4430。这项工作可以防止恶意提取言语。

采样是图形神经网络（GNN）培训的关键操作，有助于降低成本。以前的文献已经通过数学和统计方法探索了改进采样算法。但是，采样算法和硬件之间存在差距。在不考虑硬件的情况下，算法设计人员仅在算法级别优化采样，缺少通过利用硬件功能来促进现有采样算法效率的巨大潜力。在本文中，我们开创了一个为主流采样算法提出的统一编程模型，称为GNNSampler，涵盖了各个类别中采样算法的关键过程。其次，为了利用硬件功能，我们选择数据局部性作为案例研究，并在图中探索节点及其邻居之间的数据位置，以减轻采样中不规则的内存访问。第三，我们在GNNSampler中实现了各种采样算法的局部感知优化，以优化一般的采样过程。最后，我们强调在大图数据集上进行实验，以分析训练时间，准确性和硬件级指标之间的相关性。广泛的实验表明，我们的方法通用到主流采样算法，并有助于大大减少训练时间，尤其是在大规模图中。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

Learning feature interactions is the key to success for the large-scale CTR prediction and recommendation. In practice, handcrafted feature engineering usually requires exhaustive searching. In order to reduce the high cost of human efforts in feature engineering, researchers propose several deep neural networks (DNN)-based approaches to learn the feature interactions in an end-to-end fashion. However, existing methods either do not learn both vector-wise interactions and bit-wise interactions simultaneously, or fail to combine them in a controllable manner. In this paper, we propose a new model, xDeepInt, based on a novel network architecture called polynomial interaction network (PIN) which learns higher-order vector-wise interactions recursively. By integrating subspace-crossing mechanism, we enable xDeepInt to balance the mixture of vector-wise and bit-wise feature interactions at a bounded order. Based on the network architecture, we customize a combined optimization strategy to conduct feature selection and interaction selection. We implement the proposed model and evaluate the model performance on three real-world datasets. Our experiment results demonstrate the efficacy and effectiveness of xDeepInt over state-of-the-art models. We open-source the TensorFlow implementation of xDeepInt: https://github.com/yanyachen/xDeepInt.